Skip to content

crypto: limit sha1 or sha512 blocks processed at once in assembly - #80916

Closed
AskAlexSharov wants to merge 1 commit into
golang:masterfrom
AskAlexSharov:sha1-sha512-bound-asm
Closed

crypto: limit sha1 or sha512 blocks processed at once in assembly#80916
AskAlexSharov wants to merge 1 commit into
golang:masterfrom
AskAlexSharov:sha1-sha512-bound-asm

Conversation

@AskAlexSharov

Copy link
Copy Markdown

Problem

GC stop-the-world is slow if a large input is passed to sha1/sha512

Root cause: Write passes unbounded input to block(), and assembly is not
preemptible. CL 671098 bounded md5 and sha256 for #64417; sha1 and sha512 have
the same problem and were not covered.

Stop-the-world (improved)

/sched/pauses/stopping/gc:seconds, the time the runtime spends waiting for every
P to halt, worst pause while hashing a 64 MiB buffer. linux/amd64, EPYC 4344P,
GOMAXPROCS=2

                 before        after      pauses >1ms
crypto/sha1      25.166ms      197us         2 -> 0
crypto/sha512    67.109ms      1.049ms       2 -> 0

BenchmarkSTW, added here, reports the same effect as the wall time of one
runtime.GC() that overlaps the hash:

    GOMAXPROCS=2 go test -run='^$' -bench=BenchmarkSTW -count=6 -benchtime=2s crypto/sha1 crypto/sha512
    benchstat before.txt after.txt

           │ gcwait-sec/op │ gcwait-sec/op  vs base               │
    sha1        24988.7µ ± 0%    319.3µ ± 13%  -98.72% (p=0.002 n=6)
    sha512      61965.8µ ± 0%    475.8µ ± 60%  -99.23% (p=0.002 n=6)

Throughput (no degradations)

    for i in $(seq 8); do
      GOMAXPROCS=2 go test -run='^$' -bench=BenchmarkHash -count=3 -benchtime=200ms crypto/sha1 crypto/sha512
    done
    benchstat before.txt after.txt

    sha1                     sec/op        sec/op     vs base
    Hash8Bytes/New-2     57.70n ± 6%   57.78n ± 5%  ~ (p=0.649 n=24)
    Hash8Bytes/Sum-2     52.50n ± 0%   52.46n ± 0%  ~ (p=0.770 n=24)
    Hash320Bytes/New-2   174.3n ± 0%   174.3n ± 0%  ~ (p=0.888 n=24)
    Hash320Bytes/Sum-2   171.6n ± 0%   171.7n ± 0%  ~ (p=0.368 n=24)
    Hash1K/New-2         433.8n ± 0%   433.9n ± 0%  ~ (p=0.922 n=24)
    Hash1K/Sum-2         432.1n ± 0%   432.0n ± 0%  ~ (p=0.988 n=24)
    Hash8K/New-2         3.083µ ± 0%   3.083µ ± 0%  ~ (p=0.896 n=24)
    Hash8K/Sum-2         3.083µ ± 0%   3.082µ ± 0%  ~ (p=0.938 n=24)
    Hash256K/New-2       96.96µ ± 0%   96.94µ ± 0%  ~ (p=0.657 n=24)
    Hash256K/Sum-2       96.90µ ± 0%   96.96µ ± 0%  ~ (p=0.092 n=24)
    Hash1M/New-2         387.9µ ± 0%   388.0µ ± 0%  ~ (p=0.214 n=24)
    Hash1M/Sum-2         387.8µ ± 0%   387.9µ ± 0%  ~ (p=0.428 n=24)
    geomean              2.796µ        2.796µ       +0.01%

    sha512                    sec/op        sec/op     vs base
    Hash8Bytes/New-2      149.4n ± 0%   149.3n ± 0%       ~ (p=0.849 n=24)
    Hash8Bytes/Sum384-2   157.6n ± 0%   158.0n ± 0%  +0.25% (p=0.003 n=24)
    Hash8Bytes/Sum512-2   157.5n ± 0%   157.8n ± 0%       ~ (p=0.106 n=24)
    Hash1K/New-2          1.082µ ± 0%   1.084µ ± 0%       ~ (p=0.053 n=24)
    Hash1K/Sum384-2       1.089µ ± 0%   1.089µ ± 0%       ~ (p=0.329 n=24)
    Hash1K/Sum512-2       1.093µ ± 0%   1.091µ ± 0%       ~ (p=0.538 n=24)
    Hash8K/New-2          7.655µ ± 0%   7.650µ ± 0%  -0.07% (p=0.045 n=24)
    Hash8K/Sum384-2       7.668µ ± 0%   7.652µ ± 0%  -0.21% (p=0.000 n=24)
    Hash8K/Sum512-2       7.660µ ± 0%   7.649µ ± 0%  -0.14% (p=0.000 n=24)
    Hash256K/New-2        240.5µ ± 0%   239.9µ ± 0%       ~ (p=0.173 n=24)
    Hash256K/Sum384-2     240.7µ ± 0%   240.0µ ± 0%  -0.32% (p=0.024 n=24)
    Hash256K/Sum512-2     240.2µ ± 0%   240.0µ ± 0%       ~ (p=0.136 n=24)
    Hash1M/New-2          960.8µ ± 0%   962.4µ ± 0%       ~ (p=0.519 n=24)
    Hash1M/Sum384-2       961.2µ ± 0%   961.0µ ± 0%       ~ (p=0.814 n=24)
    Hash1M/Sum512-2       960.8µ ± 0%   960.5µ ± 0%       ~ (p=0.878 n=24)
    geomean               12.44µ        12.44µ       -0.04%

References:

Problem

GC stop-the-world is slow if a large input is passed to sha1/sha512

Root cause: Write passes unbounded input to block(), and assembly is not
preemptible. CL 671098 bounded md5 and sha256 for #64417; sha1 and sha512 have
the same problem and were not covered.

Stop-the-world (improved)

/sched/pauses/stopping/gc:seconds, the time the runtime spends waiting for every
P to halt, worst pause while hashing a 64 MiB buffer. linux/amd64, EPYC 4344P,
GOMAXPROCS=2

                     before        after      pauses >1ms
    crypto/sha1      25.166ms      197us         2 -> 0
    crypto/sha512    67.109ms      1.049ms       2 -> 0

BenchmarkSTW, added here, reports the same effect as the wall time of one
runtime.GC() that overlaps the hash:

```
    GOMAXPROCS=2 go test -run='^$' -bench=BenchmarkSTW -count=6 -benchtime=2s crypto/sha1 crypto/sha512
    benchstat before.txt after.txt

           │ gcwait-sec/op │ gcwait-sec/op  vs base               │
    sha1        24988.7µ ± 0%    319.3µ ± 13%  -98.72% (p=0.002 n=6)
    sha512      61965.8µ ± 0%    475.8µ ± 60%  -99.23% (p=0.002 n=6)
```

Throughput (no degradations)

```
    for i in $(seq 8); do
      GOMAXPROCS=2 go test -run='^$' -bench=BenchmarkHash -count=3 -benchtime=200ms crypto/sha1 crypto/sha512
    done
    benchstat before.txt after.txt

    sha1                     sec/op        sec/op     vs base
    Hash8Bytes/New-2     57.70n ± 6%   57.78n ± 5%  ~ (p=0.649 n=24)
    Hash8Bytes/Sum-2     52.50n ± 0%   52.46n ± 0%  ~ (p=0.770 n=24)
    Hash320Bytes/New-2   174.3n ± 0%   174.3n ± 0%  ~ (p=0.888 n=24)
    Hash320Bytes/Sum-2   171.6n ± 0%   171.7n ± 0%  ~ (p=0.368 n=24)
    Hash1K/New-2         433.8n ± 0%   433.9n ± 0%  ~ (p=0.922 n=24)
    Hash1K/Sum-2         432.1n ± 0%   432.0n ± 0%  ~ (p=0.988 n=24)
    Hash8K/New-2         3.083µ ± 0%   3.083µ ± 0%  ~ (p=0.896 n=24)
    Hash8K/Sum-2         3.083µ ± 0%   3.082µ ± 0%  ~ (p=0.938 n=24)
    Hash256K/New-2       96.96µ ± 0%   96.94µ ± 0%  ~ (p=0.657 n=24)
    Hash256K/Sum-2       96.90µ ± 0%   96.96µ ± 0%  ~ (p=0.092 n=24)
    Hash1M/New-2         387.9µ ± 0%   388.0µ ± 0%  ~ (p=0.214 n=24)
    Hash1M/Sum-2         387.8µ ± 0%   387.9µ ± 0%  ~ (p=0.428 n=24)
    geomean              2.796µ        2.796µ       +0.01%

    sha512                    sec/op        sec/op     vs base
    Hash8Bytes/New-2      149.4n ± 0%   149.3n ± 0%       ~ (p=0.849 n=24)
    Hash8Bytes/Sum384-2   157.6n ± 0%   158.0n ± 0%  +0.25% (p=0.003 n=24)
    Hash8Bytes/Sum512-2   157.5n ± 0%   157.8n ± 0%       ~ (p=0.106 n=24)
    Hash1K/New-2          1.082µ ± 0%   1.084µ ± 0%       ~ (p=0.053 n=24)
    Hash1K/Sum384-2       1.089µ ± 0%   1.089µ ± 0%       ~ (p=0.329 n=24)
    Hash1K/Sum512-2       1.093µ ± 0%   1.091µ ± 0%       ~ (p=0.538 n=24)
    Hash8K/New-2          7.655µ ± 0%   7.650µ ± 0%  -0.07% (p=0.045 n=24)
    Hash8K/Sum384-2       7.668µ ± 0%   7.652µ ± 0%  -0.21% (p=0.000 n=24)
    Hash8K/Sum512-2       7.660µ ± 0%   7.649µ ± 0%  -0.14% (p=0.000 n=24)
    Hash256K/New-2        240.5µ ± 0%   239.9µ ± 0%       ~ (p=0.173 n=24)
    Hash256K/Sum384-2     240.7µ ± 0%   240.0µ ± 0%  -0.32% (p=0.024 n=24)
    Hash256K/Sum512-2     240.2µ ± 0%   240.0µ ± 0%       ~ (p=0.136 n=24)
    Hash1M/New-2          960.8µ ± 0%   962.4µ ± 0%       ~ (p=0.519 n=24)
    Hash1M/Sum384-2       961.2µ ± 0%   961.0µ ± 0%       ~ (p=0.814 n=24)
    Hash1M/Sum512-2       960.8µ ± 0%   960.5µ ± 0%       ~ (p=0.878 n=24)
    geomean               12.44µ        12.44µ       -0.04%
```

References:
- Go stdlib sha256, md5: #64417
@google-cla

google-cla Bot commented Aug 17, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@AskAlexSharov AskAlexSharov closed this by deleting the head repository Aug 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant